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Office Action Summary 
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Examiner 

Daniel A. Nolan 



Applicant(s) 

SUOMELA ET AL. 



Art Unit 

2654 



- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- if the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )□ Responsive to communiGation(s) filed on . 

2a)^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for fornrial matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-49 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n C!aim(s) is/are allowed. 

6) 13 Claim(s) 1-49 is/are reiected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) ^ The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 19 December 2000 is/are: a)n accepted or b)KI objected to by the Examiner. 

Applicant may not request that any objection to the drawing{s) be held in abeyance. See 37 CFR 1 .85(a). 

11) 0 The proposed drawing correction filed on is: a)n approved b)^ disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)nAII b)n Some*c)n None of: 

1 Certified copies of the priority documents have been received. 

2.n Certified copies of the priority documents have been received in Application No. . 



3.n Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) D The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attach ment(5) 



1) K Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) CH Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 



4) im Interview Sunnmary (PTO-413) Paper No(s). 

5) dl Notice of Infomnal Patent Application (PTO-152) 

6) n Other: 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 04-01) 



Office Action Summary 



Part of Paper No. 9 
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DETAILED ACTION 



1 . The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 



Drawings 



2. In order to avoid abandonment, the drawing informalities noted in Paper No. 7, 
mailed on 03 June 2003, must now be corrected. Correction can only be effected in the 
manner set forth in the above noted paper. Failure to timely submit the proposed 
drawing correction will result in the abandonment of the application. 



Response to Amendment 

3. The filing of 05 September 2003 has been entered to the following effect: 

- The claims were changed as indicated and both the respective objections and the 
35 use § 1 12 rejection have been withdrawn as satisfied. 

- The claim has been examined on the merits in light of the amended limitations. 

- Claims 47-49 were added and examined on the merits. 
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Response to Arguments 



4. Applicant's arguments filed 05 September 2003 have been fully considered but 
they are not persuasive. 

- In response to applicant's argument that the references fail to show certain features 
of applicant's invention, it is noted that the features upon which applicant relies (i.e., 
speech-based error recovery) are not recited in the rejected claim(s). Although the 
claims are interpreted in light of the specification, limitations from the specification 
are not read into the claims. See In re Van Geuns, 988 F.2d 1 181 , 26 

USPQ2d 1057 (Fed. Cir. 1993). 

- With regard to the issue of the prior art of French-St. George et al teaching against a 
time limit, the argument contradicts a prior argument that the prior art fails to address 
an expiration other than the cited general description of the prior art. The argument 
acknowledges that no position is taken on the subject of time (response page 15 
lines 4-5). The prior art cannot be taken to "teach against" a feature while in the next 
instant, the same prior art is said to disregard the matter. 

- In further regard to the clarification and amendment by the Applicant making the 2^^ 
input is not restricted to speech as contrary to the understanding of the Examiner, 
the 35 use § 102 rejection is withdrawn and the examination of the entire claim has 
been revisited with regard to that feature. 
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Specification 

5. The disclosure is objected to because of the following informalities: 

- "Voiced" is misspelled (page 17 line 12). 
Appropriate correction is required. 

6. The disclosure is objected to because it contains an embedded hyperlink and/or 
other form of browser-executable code (page 17 lines 6 & 18). Applicant is required to 
delete the embedded hyperlink and/or other form of browser-executable code. 
SeeMPEP§ 608.01. 

Claim Objections 

7. Claims 27 and 40 are objected to because of the following informalities: 

- The limits of the last feature of the claim are subject to interpretation. Indenting the 
wording to reflect the structure will prevent errors of misunderstanding in the future. 
See MPEP Content of Specification, which reads in part: 

Arrangement of the Specification provided in 37 CFR 1.77(b): 

(i) Claim or Claims : See 37 CFR 1 .75 and MPEP § 608.01 (m). The claim or claims must commence on separate 

sheet (37 CFR 1 .52(b)). Where a claim sets forth a plurality of elements or steps, each element or step of the 
claim should be separated by a line indentation. There may be plural indentations to further segregate 
subcombinations or related steps. See 37 CFR 1 .75 and MPEP § 608.01 (i)-(p). 

- The Examiner is proceeding with the understanding claim 27 should be read: 

"...a speech recognition algorithm connected to said central processing unit for executing speech recognition; and 
"a primary control circuit connected to said central processing unit for 
processing said inputted and audible commands and 

activating speech recognition in response to an event for a speech recognition time period and 
deactivating speech recognition after the speech recognition time period has elapsed." 
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- The Examiner is also proceeding with the understanding claim 40 should be read: 

"...a speech recognition algorithm connected to said central processing unit for executing speech recognition; and 
" software means operative on the processor for 

maintaining in said memory unit a database identifying at least one context related word set, 

scanning for an event at the terminal, 

performing a 1^' command in response to the event, 

activating speech recognition by executing said speech recognition algorithm for a speech recognition time period in 
response to detecting said event at said terminal, 

deactivating speech recognition after the speech recognition time period has elapsed, and 
performing a 2™* command received during said speech recognition time. 

Appropriate correction is required. 



8. Claims 23, 36 and 43 contains the adjective "short". It is suggested to use a 
more precise definition. 



Claim Rejections - 35 USC § 103 



French-St George etal & Matthews 

9. Claims 1-3, 5-17, 27-32, 40, and 41 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over French-St. George al (U.S. Patent 6,01 2,030) in view of Matthews 
(U.S. Patent 4,481, 384 A). 



1 0. As per claims 1 , 2, 27, and 28 and French-St. George at al teach: 

- detecting an event at the terminal (picking up the mobile phone), and performing a 

1®* command in response to said event (turning on all default input/output modalities, 

column 6 lines 24-26 with Figure 6a); 
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- automatically activating speech recognition at the terminal in response to said event 
(inherent in "the user may then choose to complete the task by issuing a speech 
command", column 6 lines 36-38), 

- determining whether a 2"^ command is received via a secondary speech input to be 
recognized or a primary input during a speech recognition time period commencing 
the performance of said 1^* command, and deactivating speech recognition if the 2"*^ 
command is received via the primary input, and performing the 2"^ command 
(column 2 lines 47-50; column 6 lines 54-57; column 8 lines 19-22). 

While French-St. George et al teaches voice response within a period, they do 
not specify the alternative of deactivation. The voice recognizing telephone call denial 
system of Matthews (column 1 1 lines 66-68) reads on the feature of (e) deactivating the 
speech recognition at the terminal if it is determined that the 2"^ command is not 
received by the one of the speech recognition and a primary input of the terminal in said 
step (d) during the speech recognition time period; It would have been obvious to a 
person of ordinary skill in the art of speech signal processing at the time of the invention 
to apply the method/teachings of Matthews to the device/method of French-St. George 
et al so as to disassociate speech as a momentary input. 

11. As per claims 3, 29, 40, and 41 , French-St. George al teach determining a 
context in which speech recognition is activated and the corresponding applicable 
commands {"At each layer, the speech recognizer loads an appropriate vocabulary list", 
column 8 lines 5-6), inherently involving a word set database and a secondary control 
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circuit determining said context. The rest of the limitations in claims 40 and 41 are the 
same or similar to those in claims 1 , 2 , 27, and 28, above, and thus are rejected for the 
same reasons. 

1 2. As per claim 4, French-St. George et al teach a default word set comprising 
commands applicable in all contexts {"quit'\ "select other service'', "none of these", 
column 10 lines 9-15). 

13. As per claims 5, 8, and 30, French-St. George a/ teach displaying at least a 
portion of the applicable commands (e.g. column 8 line 61 through column 9 line 7). 

14. As per claim 7, French-St. George et al teach verifying the correctness of spoken 
command recognition (speech based error recovery, column 3 lines 40-43). 

1 5. As per claim 1 0-1 5, French-St. George et al teach buffering commands received 
during the speech recognition time period and peri'orming them, and repeating the steps 
(c) - (f) for the command last performed (column 6 lines 64-66; Fig. 6a). 

16. As per claims 16-17, 31, and 32, French-St. George et a/ teach the event of 
pressing a button on a mobile phone and having the primary input comprise buttons 
(column 6 lines 1-4). 
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1 7. Regarding claims 6 and 9, while French-St. George et al do not explicitly teach 
audibly outputting the applicable commands, they do teach that the speech recognizer 
provides a full range of speech prompts to the user (column 3 lines 44-45). It would 
have been obvious for an artisan at the time of invention to have these speech prompts 
include listing the applicable commands in a given context so that the user does not 
have to look at a display or remember what these commands are. 

18. Regarding claims 47, 48 and 49; the claims are set forth with the same limits as 
claims 1 , 27 and 40, respectively. The feature of receipt of information at the terminal 
from the environment of the terminal and notification of an external event is implicitly 
disclosed by French-St. George et al (column 6 lines 27-30) where the information is 
provided that a modality selection was made. 



French-St- George et a/, Matthews & Jenkins 

19. Claims 18-26, 33-39, and 42-46 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over French-St. George et al & Matthews as applied to claims 1 , 2, 27, 
and 28, above, in view of Jenkins (U.S. Patent 6,377,793). 

French-St. George et a/ teach that their multimodal interface is "particularly useful 
in hand held communications devices such as mobile phones and other specialized or 
portable terminals for information access" (column 7 lines 27-30), but do not specifically 
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mention a personal digital assistant or wearable computers or connecting via short 
range radio technology or connecting to the internet. 

However, Jenkins teaches that such communications devices include "personal 
communication devices, communication enabled wearable computers, personal digital 
assistants, ... , internet appliances" as well as "any other body-worn or user-supported 
computer device which is capable of wireless communication" (column 5 lines 31-36 
and 39-41 ). Therefore, it would have been obvious for an artisan at the time of 
invention to use the French-St. George et al interface therein and to make the defining 
event the connection to the relevant information source, including by phone call and a 
short message, so as to make these devices more versatile and user-friendly for 
receiving information. 



Conclusion 



20. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

- Brenig (U.S. Patent 4,426,733 A) voice-controlled operator-interacting radio 
transceiver. 

- Kishi et al (U.S. Patent 4,450,545 A) voice responsive door lock. 

- Vander Molen (U.S. Patent 4,520,576 A) conversational voice command control for 
home appliance. 

- Fuiii et al (U.S. Patent 4,885,791 A) speech recognition apparatus. 
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- Metroka et al (U.S. Patent 5,175,759 A) connmunications device with movable 
element control interface. 

- Cohrs et al (U.S. Patent 5,930,751 A) implicit confirmation for automatic speech 
recognition. 

21 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

22. Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Daniel A. Nolan at telephone (703) 305-1368 whose 
normal business hours are Mon, Tue, Thu & Fri, from 7 AM to 5 PM. 
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If attempts to contact the examiner by telephone are unsuccessful, supervisor 
Richemond Dorvil can be reached at (703)305-9645. 

The fax phone number for Technology Center 2600 is (703)872-9314. Label 
informal and draft communications as "DRAFT" or "PROPOSED", & designate formal 
communications as "EXPEDITED PROCEDURE". Formal response to this action may 
be faxed according to the above instructions, 
or mailed to: Box AF 



Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to Technology Center 2600 Customer Service Office at 
telephone number (703) 306-0377. 



Commissioner of Patents and Trademarks 
Washington, D.C. 20231 



or hand-delivered to: 



Crystal Park 2, 

2121 Crystal Drive, Arlington, VA, 
Sixth Floor (Receptionist). 



Daniel A. Nolan 
Examiner 
Art Unit 2654 



DAN/d 

October 13, 2003 




/ RICHEMOND DORVIL 
SUPERVISORY PATENT EXAMINER 



